Create a Process Tracing Protocol

You are a senior MEAL specialist with expertise in qualitative causal inference methods. Your task is to create a process tracing protocol for evaluating whether and how a program caused an observed outcome.

The evaluation involves a single case or small number of cases where statistical comparison is not feasible. Process tracing is appropriate because the evaluation needs to establish whether the program was a necessary or sufficient cause of the observed change.

Develop the following components:

1. Causal Hypothesis Formulation:

   • Primary hypothesis: The specific causal claim to be tested
   • Alternative hypotheses (at least 3): Other plausible causal explanations
   • Null hypothesis: The outcome would have occurred without the program
   • For each hypothesis, specify the causal mechanism (the step-by-step process through which the cause is expected to produce the effect)

2. Causal Mechanism Mapping:

   • Break down the primary hypothesis into a sequence of 4-7 causal steps
   • For each step in the mechanism:
     • What entity is involved
     • What action or transmission occurs
     • What observable evidence would confirm this step occurred
     • What the absence of evidence would mean
   • Create a mechanism diagram showing the causal chain

3. Evidence Tests Design: For each key step in the causal mechanism, design diagnostic tests using Beach and Pedersen's four test types:

   • Straw-in-the-wind tests: Evidence that is consistent with the hypothesis but not confirmatory (neither necessary nor sufficient)
   • Hoop tests: Evidence that must be present for the hypothesis to survive (necessary but not sufficient)
   • Smoking gun tests: Evidence that strongly confirms the hypothesis if found (sufficient but not necessary)
   • Doubly decisive tests: Evidence that both confirms the hypothesis and eliminates alternatives (both necessary and sufficient)
   • For each test: describe the specific evidence sought, classify the test type, specify the source, and state what passing or failing the test means for the hypothesis

4. Evidence Collection Plan:

   • Evidence inventory: What types of evidence are needed (documents, testimony, observational, physical)
   • Source mapping: Who has this evidence, where it is located, how to access it
   • Prioritization: Which evidence tests are most diagnostic and should be pursued first
   • Timeline and sequencing of evidence collection
   • At least 10 specific pieces of evidence to seek, with their test classification

5. Confidence Updating Framework:

   • Prior confidence levels for each hypothesis (before evidence collection)
   • Bayesian-inspired updating approach: How each piece of evidence shifts confidence toward or away from each hypothesis
   • Confidence scale (e.g., very low, low, moderate, high, very high) with threshold definitions
   • Evidence tracking matrix with columns: Evidence Item, Test Type, Result (passed/failed/inconclusive), Confidence Shift (direction and magnitude), Updated Confidence Level

6. Alternative Explanation Assessment:

   • For each alternative hypothesis, specify the evidence tests that would confirm or eliminate it
   • Interaction effects: Can the program and alternative causes both be contributing causes? How to assess their relative contributions
   • Equifinality consideration: Could multiple causal paths lead to the same outcome?

7. Reporting Template:

   • Narrative causal account structure
   • Evidence summary table
   • Final confidence assessment for each hypothesis
   • Limitations and caveats
   • "Weight of evidence" conclusion format

Output Format:

Deliver all components as clearly labeled sections. Evidence tests should be formatted as a detailed table. The causal mechanism should be presented as both a narrative sequence and a visual diagram description. The confidence updating framework should include a worked example showing how evidence shifts confidence.