Design an Evaluation Matrix
Design an evaluation matrix linking questions to methods, sources, and analysis.
Draft: Design an Evaluation Matrix
Your Prompt
- What You Need
- Provide substantive evaluation questions or TOR, program logic, intended users, source availability, constraints and relevant standards.
- Choose This Prompt When
- Use this to map how evaluation questions will be answered. Choose Review an Analysis Plan to assess an existing analytical approach, or Draft an Evaluation Inception Report for the larger design document.
Describe your goal, audience, available information and constraints. Your words are added to the prompt as supplied. You can also use the guided fields below.
Kept in this tab while you work. Copy and Start includes this description.
Copy this prompt and paste it into your AI chat to start. If it needs source material, the AI will ask for it.
Evidence and applicability: use only supplied or verifiable evidence for historical facts, values, quotations, completed activities and approvals. Preserve conflicting sources with their provenance; do not choose a preferred value without adjudication. Mark missing evidence as unknown or not evidenced, not as proof an activity never occurred. Keep proposed plans and illustrative examples separate from findings. Never invent a baseline, quote, pilot, validation or sign-off to complete a template. Match methods to the purpose and population; a suitable single-method design is valid. Distinguish descriptive change from causal attribution. Check obligations in the applicable funding agreement rather than assuming a universal donor rule.
You are a senior MEAL specialist designing an evaluation matrix for your program, using criteria selected for its actual purpose and TOR, rather than an assumed standard checklist.
The matrix is the operational backbone of the evaluation: it shows how each evaluation question will be answered.
Requirements:
- Evaluation questions. List only the priority evaluation questions needed for the intended use. Each must be answerable, not rhetorical.
- Sub-questions and judgment criteria. For each question, define useful sub-questions without a fixed count and proposed judgment criteria for assessing performance, using approved standards where supplied and marking new thresholds for agreement.
- Methods. Specify the methods for each question (e.g., document review, KIIs, FGDs, survey, secondary data analysis). Use more than one source or method only where it adds defensible value; a suitable single-method design is valid.
- Data sources. Identify the specific data sources and informant categories.
- Analysis approach. State how data from each method will be analyzed and how results will be brought together (e.g., qualitative coding frame, descriptive statistics, contribution analysis).
- Limitations and mitigation. For each question, flag the most likely limitations (e.g., access, recall bias, sample size) and how they will be mitigated.
Output Format:
Produce:
- A 1-paragraph framing.
- An evaluation matrix table with columns: evaluation question, sub-questions, judgment criteria, methods, data sources, analysis approach, limitations/mitigation.
- A short synthesis note explaining integration or corroboration where relevant, how conflicting evidence will be retained and examined, and any justified single-method choices.
- A list of inputs the evaluation team will need to begin (documents, contacts, datasets).
Inputs and draft status: Request the substantive TOR or evaluation questions, program logic, intended users, actual available data sources, access constraints and applicable standards. A TOR title or list of source names is insufficient to establish answerability. If these are missing, provide a provisional structure and decisions needed; do not invent source availability, thresholds, sample sizes or approvals. This design matrix proposes how questions will be answered; it does not report findings.
Check: Score It with the Methodology Rigor Rubric
The Review Prompt
Paste the draft at the bottom of this prompt and run it. It scores each criterion from 1 to 5 and says why.
Establish the document purpose, design, stage, applicable requirements, and available source evidence before rating. Distinguish N/A with a reason, not evidenced, unresolved conflicting evidence, and observed inadequate performance. Missing records do not prove an activity did not occur. Missing records alone justify evidence requests and a deferred readiness decision, not an inferred failure or mandatory repeat activity. Do not invent documentary, signature or approval requirements. Cite document locations for every finding. Keep supported facts separate from proposals, assumptions, and illustrative examples. Never invent completed fieldwork, validation, consent, approval, values, dates, quotations, or assigned ownership. Label unconfirmed owners and proposed revisions for confirmation. Report blocking ethics, safety, confidentiality, measurement-validity, or inference failures first regardless of scores. Give a total and band only when all five dimensions are applicable and sufficiently evidenced with no unresolved evidence conflict; otherwise show the dimension profile, coverage, and evidence requests without a total, band, or rescaled equivalent. A documentation score never authorizes use, fielding, compliance, or approval.
You are an expert M&E methodologist with experience across both quantitative and qualitative methods. Score the methodology section of the document I will provide using the rubric below. The document may be a MEL plan, evaluation ToR, inception report, evaluation report, contribution analysis, needs assessment, sampling plan, or any document where methodology choices need justification and quality safeguards.
SCORING RUBRIC - Methodology Rigor Rate each applicable, evidenced dimension 1-5. Choose the highest anchor fully supported by the evidence; a material defect takes precedence over a favorable element count. An absent required feature in a complete submitted artifact is an observed documentation defect; an unavailable source or record is not evidenced, not proof the underlying activity failed.
DIMENSION 1: Methodology Justification and Method-Question Fit
- Score 5: All 4 elements: Methodology is explicitly justified relative to the research questions or evaluation questions (not just named). Each major method maps to specific questions it is designed to answer. Alternatives have been considered and rejected with documented reasoning. The choice is appropriate to the context (operating environment, data availability, ethical constraints, capacity), not just academic best practice in the abstract.
- Score 4: Three of four elements adequately specified; the remaining element incomplete: Methodology justified and mapped to questions; alternatives or context considerations partial.
- Score 3: Methodology named and broadly tied to questions but justification is thin. Alternatives not considered. Context fit assumed rather than examined.
- Score 2: Methodology named without justification. No method-question mapping.
- Score 1: No methodology described or methodology disconnected from research questions.
DIMENSION 2: Sampling and Data Source Adequacy
- Score 5: Sample or source coverage has a rationale suited to the questions: precision and uncertainty for descriptive estimates, power and effect size for planned contrasts where relevant, qualitative information or case-coverage logic, or source-inventory coverage. Selection, eligibility, frame or recruitment source, and inference limits are explicit.
- Score 4: The design-appropriate adequacy rationale and selection method are clear; a minor eligibility, frame, or coverage detail remains.
- Score 3: Selection is described but adequacy rationale or material coverage limits need clarification.
- Score 2: Coverage or selection is poorly justified for the intended claims, leaving a material inference risk.
- Score 1: No usable selection or coverage rationale exists, or the approach cannot support the claims. No universal power or saturation calculation is required.
DIMENSION 3: Triangulation and Multi-Method Strategy
- Score 5: All 4 elements: The selected evidence strategy fits the questions and claims. Necessary cross-checks are specified. Single-method safeguards or justified integration are explicit. Sources and methods have distinct, proportionate roles.
- Score 4: Most applicable elements: Strategy fits the questions; one safeguard, cross-check, or integration role is partially documented.
- Score 3: Strategy is named but its relation to claims, safeguards, or synthesis is unclear.
- Score 2: Chosen methods do not address the questions adequately or duplicate effort without a justified role.
- Score 1: Method choices are unsupported and consequential limitations ignored. A justified single-method design can earn 5.
DIMENSION 4: Data Quality Safeguards
- Score 5: All 4 elements: Quality assurance procedures documented (supervision, spot checks, back-translation if relevant, audit trails). Consistency checks specified where relevant (inter-rater reliability for coding, double entry for surveys). Pretesting or piloting is planned and described for any primary data collection. Data cleaning protocols specified (rules for handling outliers, missing data, contradictory responses).
- Score 4: Three of four elements adequately specified; the remaining element incomplete: QA procedures and pretesting documented; consistency checks or cleaning partial.
- Score 3: Some QA procedures named but not operationalized. Pretesting mentioned but not described. Cleaning protocols absent.
- Score 2: Generic mention of "quality" without specific procedures. No pretesting plan.
- Score 1: No data quality safeguards described.
DIMENSION 5: Transparency, Limitations, and Bias Mitigation
- Score 5: All 4 elements: Limitations are explicitly stated and specific (not generic "our study has limitations"). Biases are named with concrete mitigation strategies (e.g., "selection bias from voluntary participation will be mitigated by also interviewing non-participants"). Assumptions are documented (what we assumed to be true that affects findings). Procedures and reasoning are auditable at a level appropriate to the analytic tradition; identical interpretive findings across qualitative researchers are not required.
- Score 4: Three of four elements adequately specified; the remaining element incomplete: Limitations and biases named; mitigation or replicability partial.
- Score 3: Generic limitations stated. Biases mentioned without mitigation. Assumptions implicit.
- Score 2: Limitations are formulaic boilerplate. No bias acknowledgment.
- Score 1: No limitations, biases, or assumptions documented.
OUTPUT FORMAT: Return your assessment as a table followed by a summary:
| Dimension | Score (1-5), N/A, or Not Evidenced | Evidence from Document | Priority Revision |
|---|---|---|---|
| Methodology Justification and Method-Question Fit | |||
| Sampling and Data Source Adequacy | |||
| Triangulation and Multi-Method Strategy | |||
| Data Quality Safeguards | |||
| Transparency, Limitations, and Bias Mitigation |
First report blocking issues, unresolved conflicts, applicability and evidence coverage. Only if all five dimensions are applicable and sufficiently evidenced with no unresolved source conflict, report Total: X/25 and Band: Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10), as a secondary documentation summary. Otherwise omit both total and band; report the dimension profile and evidence requests without rescaling. Single Most Important Revision: [One specific sentence]
For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.
DOCUMENT TO SCORE: [Paste your methodology section or full document here]
Fix: Revise What Scored Low
The Revise Prompt
Paste the draft and its scores at the bottom of this prompt. It revises the parts that scored below 4.
Establish the document purpose, design, stage, applicable requirements, and available source evidence before rating. Distinguish N/A with a reason, not evidenced, unresolved conflicting evidence, and observed inadequate performance. Missing records do not prove an activity did not occur. Missing records alone justify evidence requests and a deferred readiness decision, not an inferred failure or mandatory repeat activity. Do not invent documentary, signature or approval requirements. Cite document locations for every finding. Keep supported facts separate from proposals, assumptions, and illustrative examples. Never invent completed fieldwork, validation, consent, approval, values, dates, quotations, or assigned ownership. Label unconfirmed owners and proposed revisions for confirmation. Report blocking ethics, safety, confidentiality, measurement-validity, or inference failures first regardless of scores. Give a total and band only when all five dimensions are applicable and sufficiently evidenced with no unresolved evidence conflict; otherwise show the dimension profile, coverage, and evidence requests without a total, band, or rescaled equivalent. A documentation score never authorizes use, fielding, compliance, or approval.
You are an expert M&E methodologist. Review the methodology section I will provide and fix every problem that falls below the Good (4) threshold on the Methodology Rigor rubric.
Separate supported corrections from evidence requests and proposed text requiring confirmation. A plausible owner, date, rationale, or account of an activity is not a verified fact. Preserve unresolved source conflicts.
WHAT TO LOOK FOR AND FIX:
Justification problems: Methodology named without justification. No method-question mapping. Alternatives not considered. Context fit assumed.
Sampling problems: Sample size unjustified. Sampling method mismatched to question type. Inclusion criteria implicit. Sample frame undocumented.
Triangulation problems: Evidence strategy does not support the claims, necessary safeguards are missing, or integration is asserted without a justified role. A fit-for-purpose single method is not itself a defect.
Quality Safeguard problems: No QA procedures. No consistency checks. No pretesting plan. No data cleaning protocols.
Transparency problems: Generic limitations. Biases unnamed or unmitigated. Assumptions implicit. Replication not feasible.
OUTPUT FORMAT: For each problem found:
- Dimension: [which dimension]
- Location: [which section of the document]
- Problem: [one sentence describing what is wrong]
- Revised version: [the corrected text]
Then provide the complete revised methodology section incorporating all corrections.
DRAFT TO REVISE: [Paste your methodology section here]