At a Glance
Side-by-Side Comparison
| Factor | Outcome Harvesting (OH) | Most Significant Change (MSC) |
|---|---|---|
| Developed by | Ricardo Wilson-Grau (developed from ~2002, documented 2015) | Rick Davies and Jess Dart (2005 guide) |
| Starting point | Observed outcomes, traced back to attribution | Stories of change collected from participants |
| Driven by | Evaluator/facilitator, with informant input | Participants and selection panel members |
| Output format | Structured outcome statements: description, significance, contribution, evidence | Narrative stories ranked by stakeholder panels |
| Typical timeline | 4-8 weeks for the evaluation cycle | 3-6 months for a full cycle with selection panels |
| Rigor comes from | Explicit outcome statements, contribution evidence, and substantiation of a transparently selected sample | Selection panels: multi-level filtering by stakeholders who justify their choices |
| Participant voice | Moderate (informant interviews, not participant-authored) | High (participants author stories in their own words) |
| Accountability reporting fit | Strong (structured outputs work for donor reports) | Moderate (stories need structuring for donor outputs) |
| Best for | Complex/emergent programs, advocacy, systems change, innovation | Participatory programs, empowerment, community-led, unanticipated-change contexts |
| Facilitator skill required | High (substantiation design, evidence triangulation) | High (selection panel design, facilitation across levels) |
| Cost | Moderate: 30-60% of a standard summative evaluation | Moderate to high: 40-80% of a standard summative evaluation |
For the formal method definitions, see outcome harvesting and most significant change.
What Do Outcome Harvesting and Most Significant Change Have in Common?
Both methods share three foundational features that distinguish them from counterfactual-based evaluation approaches.
Both are theory-based, not counterfactual-based. Neither tries to construct a "what would have happened without the program" comparison. Instead, both ask "what change is observable and how is the program implicated?" This makes both appropriate when counterfactual design is infeasible (small-n, ethical issues, emergent outcomes) or inappropriate (complex programs where no control group is meaningful). See counterfactual and theory-based evaluation.
Both are qualitative-primary. Both work in the language of narrative and description rather than statistics and effect sizes. Both produce outputs that can be supplemented by quantitative data but do not depend on quantitative measurement for their primary findings.
Both emphasize learning alongside accountability. Neither method is a pure compliance tool. Both are designed to produce findings the program can actually use, including unanticipated outcomes that conventional M&E would miss.
These shared foundations mean both methods fit naturally with adaptive management, learning agendas, and theory of change reasoning. See theory of change for the broader framework.
When to Use Each
When Is Outcome Harvesting the Better Choice?
Outcome Harvesting is the better choice when the following apply.
The program is complex or emergent. Advocacy programs, governance reform work, systems-change initiatives, and innovation projects produce outcomes that were not fully predictable at design. OH handles this well: it starts with what actually happened rather than what was planned, and traces back to program contribution. Programs with highly specific theories of change (service delivery, targeted behavior change) may find OH less distinctive than MSC or conventional methods.
Timeline is tight. A typical OH cycle is 4-8 weeks: identify outcomes, draft statements, substantiate with informants, finalize. MSC cycles are 3-6 months because of selection panels. For a report due in three months, cost and schedule either method against the required participation, substantiation, selection, and feedback rather than excluding MSC by duration alone.
Multiple actors contributed to outcomes. OH's contribution-analysis framing is well-suited to programs where the program shares credit with others (advocacy coalitions, policy reform work, multi-stakeholder initiatives). The outcome statement format (description, contribution, evidence) forces honesty about the program's specific share of contribution.
Accountability output is the primary deliverable. OH produces structured outcome statements that can be tabulated, counted, and aggregated. A donor report with "28 documented outcomes, 18 with strong substantiation" works. MSC's story format is harder to tabulate without additional synthesis work.
When Is Most Significant Change the Better Choice?
Most Significant Change is the better choice when the following apply.
Participant voice is central to the program's theory of change. Empowerment programs, localization-led programs, community-driven development, and participatory programs have explicit commitments to centering participant perspectives. MSC operationalizes this through participant-authored stories and participant-led selection. OH can include participant voice but does not require it structurally.
Unanticipated outcomes are expected. MSC's open-ended story prompt (typically "In the past X months, what was the most significant change you observed in Y?") invites participants to name outcomes the evaluator did not anticipate. OH is better at structuring known outcomes; MSC is better at surfacing unknown ones.
The evaluation has time for selection panels. MSC's defining feature is explicit participatory story selection and feedback; the number of levels should fit the organization and purpose. In a multi-level design: stories collected, then field-level panels select, then management-level panels select, then stakeholder/board panels select. Each level must justify its selection. Allow time for transparent selection, documented reasons, verification, and feedback. Three to six months is illustrative, not a minimum; a single-level adaptation can be appropriate when explained.
Learning is the primary purpose. MSC stories move people. They work well in internal learning events, external communications, and advocacy with audiences who respond to narrative. If the evaluation output is meant to change how internal staff think about the program or how external audiences perceive it, MSC stories do this better than OH's structured statements.
Using Them Together
Some programs combine OH and MSC. The following workflow is illustrative; collection and selection counts should follow purpose, coverage, and resources:
Story Collection (MSC Phase): solicit change stories from participants and frontline staff using MSC prompts. Explain whose perspectives will be sought and the collection scope.
Selection (MSC Phase): use a justified single-level or multi-level participatory selection process. Document significance criteria, selection reasons, whose views were represented, verification, and feedback.
Outcome Harvesting from Stories (OH Phase): rather than going to informants cold, use the selected MSC stories as the starting point. Draft outcome statements from each story (description, significance, contribution, evidence).
Substantiation (OH Phase): select outcomes for substantiation using documented risk, significance, and coverage criteria, and seek knowledgeable sources independent of the original statement author. Record the selection and evidence for each checked outcome.
Synthesis: organize outcome statements into thematic clusters, produce report.
Estimate the combined cycle and cost from the actual facilitation, collection, selection, substantiation, feedback, and synthesis tasks. Combining methods can be useful when both participant-valued significance and structured outcome statements serve the evaluation questions; use one method when it meets the purpose.
The Processes
Outcome Harvesting Process
The standard OH process has six steps, developed by Wilson-Grau.
Design (1-2 Weeks): define the harvest question (what kinds of outcomes matter), the time frame, the informants to contact. Typical harvest question: "What changes in the practices or policies of key actors has the program contributed to in the past 24 months?"
Gather Outcome Descriptions (2 Weeks): review program documents, interview informants, collect candidate outcomes. Each candidate is a draft outcome statement.
Formulate Outcome Statements (1 Week): refine each outcome into the standard format: description (what changed, when, where, who), significance (why this matters), contribution (how the program is implicated).
Substantiate (1-2 Weeks): substantiate a justified sample with knowledgeable independent sources. Identify what was checked, how it was selected, and what remains a reported claim rather than verified evidence; do not imply all outcomes were substantiated.
Analyze (1 Week): cluster outcomes by theme, identify patterns, draft findings.
Support Use (1 Week): present findings, support the program team in drawing decisions from them.
Total typical cycle: 6-9 weeks. Skilled facilitator-led. Budget for substantiation is critical; an OH without substantiation is a list of claims.
Most Significant Change Process
The Davies and Dart MSC guide describes ten steps; the core sequence below condenses them.
Establish Domains of Change (2 Weeks): identify 3-5 domains the program is trying to influence. Domains are broad categories (e.g., "changes in how women participate in community decisions"), not specific indicators.
Collect Stories (4-6 Weeks): ask participants and stakeholders for stories of the most significant change they have observed in each domain. Standard prompt: "Looking back over the last X months, what do you think was the most significant change in [domain]?"
Field-Level Selection (1 Week): field teams select the most significant story per domain, with written justification for the selection.
Program-Level Selection (1 Week): program-level panel reviews field selections, selects most significant story per domain, again with justification.
Stakeholder-Level Selection (1 Week): higher-level panel (board, management, donor if appropriate) selects most significant story.
Feedback Loops (2 Weeks): share selected stories and selection reasoning with participants, field staff, and program teams. This is where MSC's participatory learning happens.
Verification (Optional, 2 Weeks): verify key claims in selected stories with secondary sources.
Synthesis: report on selected stories, selection reasoning, and thematic patterns.
Total typical cycle: 3-6 months. Selection panel design is the core craft; bad selection design produces MSC that feels like "favorite stories" rather than rigorous qualitative evaluation.
Sector Examples
Sector Examples
The scenarios below are illustrative. Their figures, costs, and review outcomes are teaching assumptions, not reported studies or evidence of donor approval.
Health: Advocacy Coalition on Tobacco Control, Southern Africa
A regional coalition working on tobacco-control policy advocacy used Outcome Harvesting for its 3-year evaluation. OH was the right fit because: outcomes were policy-level changes contributed to by multiple coalitions and government actors; pre-specified indicators would have missed the policy wins that happened in unexpected domains; timeline for donor reporting was 8 weeks. The harvest produced 34 outcome statements across 4 categories (national policy, sub-national regulation, industry compliance, public awareness). 22 were substantiated with government documents, news reports, or partner interviews. The donor accepted OH as appropriate for advocacy evaluation; the coalition used the output to plan the next phase.
Education: Girls' Education Empowerment Program, South Asia
A 4-year girls' education program used Most Significant Change because participant voice was central to the program's empowerment theory of change. Story collection spanned 600 girls and 300 family members across 60 communities. Field-level selection panels (community committees) chose 20 stories per community; program panel narrowed to 60; stakeholder panel (board plus external reviewers) selected 12 as most significant. The process took 5 months. Selected stories revealed an unanticipated outcome: mothers reporting their own changed behavior in supporting older daughters' education. This outcome had not been in the original theory of change and prompted a program design adjustment for year 5.
Governance: Public Accountability Program, West Africa
A 3-year governance program combined OH and MSC. MSC collected 140 stories from citizens, frontline officials, and civil society partners. Field and program panels selected 45; one stakeholder panel did final selection. Then the evaluation team converted the 45 selected stories into 38 OH outcome statements (some stories clustered into single outcomes), substantiated 30 through secondary sources, and produced a synthesis report. Combined cycle was 4.5 months. The combined approach gave the donor accountability-ready outputs and gave the program participatory learning material for its next phase.
Livelihoods: Pastoralist Livelihoods Program, Sahel
A livelihoods program chose Outcome Harvesting over MSC because its theory of change was relatively specific (livelihood diversification, savings group adoption, livestock vaccination uptake) and the program management needed fast evaluation turnaround for a donor review. OH produced 24 outcome statements across 3 program components in 7 weeks. 19 were substantiated through program records, community interviews, and market data. The evaluation team flagged 5 outcomes where substantiation was weak and recommended further monitoring. OH was the right call for this program; MSC would have produced richer stories but not fit the timeline.
Mistakes & Checklist
Common Mistakes
Mistake 1: Choosing OH or MSC as a shortcut for impact evaluation. Both methods are distinct tools for different questions, not cheap alternatives to RCT or quasi-experimental design. Programs that pick OH or MSC because "impact evaluation is too expensive" often end up with evaluations that cannot answer the impact question they were trying to avoid. See RCT vs quasi-experimental for when counterfactual is actually required.
Mistake 2: Underbudgeting facilitation. Both methods depend on skilled facilitator discipline. OH without rigorous substantiation is a list of claims; MSC without rigorous selection panel design is favorite-stories. Plan relevant facilitation training, supervision, and resources; seniority alone does not establish competence.
Mistake 3: Losing transparent selection and feedback in MSC. Participants should discuss significance, record the reasons for selection, and receive feedback. Choose the number of selection levels for the organization and purpose; a justified single-level adaptation can retain these features. Extra levels alone do not establish rigor.
Mistake 4: Claiming rigor without substantiation in OH. Explain the substantiation sampling strategy and distinguish substantiated outcomes from reported claims and unresolved evidence. Not every harvested outcome must be checked, but unchecked statements must not be described as verified. Reports that present all outcomes as equally verified undermine credibility.
Mistake 5: Treating OH and MSC as interchangeable. They answer different questions. OH asks "what changed and how did the program contribute?" MSC asks "what mattered most to participants about what changed?" These are related but distinct questions, and picking the wrong method for the question produces unsatisfying output.
Mistake 6: Failing to frame donor-facing purpose. OH and MSC are not accepted equally by all donors. Proposals using either method should explicitly frame why counterfactual design is not suitable (complexity, ethics, small-n, emergent outcomes), how rigor is achieved (substantiation, selection panels, triangulation), and what the expected output structure is. Unframed, either method can come across as insufficient to traditional donors.
OH vs MSC Selection Checklist
Run through this during evaluation planning to decide which method fits.
Program characteristics:
- Program type: service delivery / advocacy / systems change / empowerment / innovation (flag applicable)
- Theory of change: specific and pre-defined or emergent and unpredictable
- Participant voice centrality: core to theory of change or supplementary
- Multi-actor contribution: single-program or multi-stakeholder
Evaluation constraints:
- Timeline available: 4-8 weeks (OH feasible) vs 3-6 months (MSC feasible)
- Budget for facilitation: experienced OH or MSC facilitator secured
- Donor acceptance: method pre-cleared with donor if required
Output requirements:
- Primary output format needed: structured outcome statements (OH) or narrative stories (MSC)
- Accountability vs learning: which is primary purpose
- Audience for findings: donor report, internal learning, external communication
Mixed-method decision:
- If OH + MSC combined: budget supports 1.3-1.6x cost vs single method
- Combined timeline of 3-4 months available
For the broader methodology selection, see how to choose evaluation methodology and qualitative vs quantitative vs mixed methods. For the theory-based evaluation framework both methods sit within, see theory-based evaluation. For current document workspaces, see the Playbooks catalog.