Design an Evaluation Matrix
Design an evaluation matrix linking questions to methods, sources, and analysis.
Draft: Design an Evaluation Matrix
Fill In Your Details (4 Blanks)
Add your information to get a tailored prompt. You can also use it as is.
Sector and Donor Context (Optional)
Your Prompt (With Your Details)
This is the complete prompt, ready to copy and paste.
You are a senior MEAL specialist designing an evaluation matrix for your program, assessed against standard evaluation criteria.
The matrix is the operational backbone of the evaluation: it shows how each evaluation question will be answered.
Requirements:
- Evaluation questions. List the prioritized evaluation questions (typically 5-8). Each must be answerable, not rhetorical.
- Sub-questions and judgment criteria. For each question, define 2-4 sub-questions and the judgment criteria that will determine 'good' vs. 'weak' performance.
- Methods. Specify the methods for each question (e.g., document review, KIIs, FGDs, survey, secondary data analysis). At least one question must triangulate two or more methods.
- Data sources. Identify the specific data sources and informant categories.
- Analysis approach. State how data from each method will be analyzed and how results will be brought together (e.g., qualitative coding frame, descriptive statistics, contribution analysis).
- Limitations and mitigation. For each question, flag the most likely limitations (e.g., access, recall bias, sample size) and how they will be mitigated.
Output Format:
Produce:
- A 1-paragraph framing.
- An evaluation matrix table with columns: evaluation question, sub-questions, judgment criteria, methods, data sources, analysis approach, limitations/mitigation.
- A short note on triangulation strategy across the full matrix.
- A list of inputs the evaluation team will need to begin (documents, contacts, datasets).
Check: Score It with the Methodology Rigor Rubric
The Review Prompt
Paste the draft at the bottom of this prompt and run it. It scores each criterion from 1 to 5 and says why.
You are an expert M&E methodologist with experience across both quantitative and qualitative methods. Score the methodology section of the document I will provide using the rubric below. The document may be a MEL plan, evaluation ToR, inception report, evaluation report, contribution analysis, needs assessment, sampling plan, or any document where methodology choices need justification and quality safeguards.
SCORING RUBRIC - Methodology Rigor Score each dimension 1-5 using these criteria:
DIMENSION 1: Methodology Justification and Method-Question Fit
- Score 5: All four elements present. Methodology is explicitly justified relative to the research questions or evaluation questions (not just named). Each major method maps to specific questions it is designed to answer. Alternatives have been considered and rejected with documented reasoning. The choice is appropriate to the context (operating environment, data availability, ethical constraints, capacity), not just academic best practice in the abstract.
- Score 4: At least three of four elements present. Methodology justified and mapped to questions; alternatives or context considerations partial.
- Score 3: Methodology named and broadly tied to questions but justification is thin. Alternatives not considered. Context fit assumed rather than examined.
- Score 2: Methodology named without justification. No method-question mapping.
- Score 1: No methodology described or methodology disconnected from research questions.
DIMENSION 2: Sampling and Data Source Adequacy
- Score 5: All four elements present. Sample size or data source coverage is justified (statistical power calculation for quantitative, saturation logic for qualitative, comprehensive coverage logic for document review). Sampling method is appropriate to the question type (purposive for explanatory, probability for prevalence, etc.). Inclusion and exclusion criteria are clear and operational. Sample frame is documented (where the sample is drawn from, how it is constructed).
- Score 4: At least three of four elements present. Sample size justified and method appropriate; inclusion criteria or sample frame partial.
- Score 3: Sampling described but justification weak. Inclusion criteria implicit. Sample frame partially documented.
- Score 2: Sampling described with minimal justification. Coverage adequacy unclear.
- Score 1: No sampling rationale or sample frame.
DIMENSION 3: Triangulation and Multi-Method Strategy
- Score 5: All four elements present. Multiple data sources or methods are used for key research questions (not just for the overall study). Triangulation strategy is explicit (which finding will be cross-checked with which source, when, how). Different methods address different aspects of the question (e.g., quantitative for prevalence, qualitative for mechanism). Sources or methods complement each other rather than duplicating coverage.
- Score 4: At least three of four elements present. Multiple methods used; triangulation strategy partial or some methods overlap.
- Score 3: Multiple methods present but triangulation logic vague ("we will triangulate"). Methods overlap on coverage.
- Score 2: Single method or two methods used redundantly. No triangulation strategy.
- Score 1: Single method without acknowledgment of its limitations or need for triangulation.
DIMENSION 4: Data Quality Safeguards
- Score 5: All four elements present. Quality assurance procedures documented (supervision, spot checks, back-translation if relevant, audit trails). Consistency checks specified where relevant (inter-rater reliability for coding, double entry for surveys). Pretesting or piloting is planned and described for any primary data collection. Data cleaning protocols specified (rules for handling outliers, missing data, contradictory responses).
- Score 4: At least three of four elements present. QA procedures and pretesting documented; consistency checks or cleaning partial.
- Score 3: Some QA procedures named but not operationalized. Pretesting mentioned but not described. Cleaning protocols absent.
- Score 2: Generic mention of "quality" without specific procedures. No pretesting plan.
- Score 1: No data quality safeguards described.
DIMENSION 5: Transparency, Limitations, and Bias Mitigation
- Score 5: All four elements present. Limitations are explicitly stated and specific (not generic "our study has limitations"). Biases are named with concrete mitigation strategies (e.g., "selection bias from voluntary participation will be mitigated by also interviewing non-participants"). Assumptions are documented (what we assumed to be true that affects findings). Replication is feasible: another researcher could reproduce key findings from what is documented.
- Score 4: At least three of four elements present. Limitations and biases named; mitigation or replicability partial.
- Score 3: Generic limitations stated. Biases mentioned without mitigation. Assumptions implicit.
- Score 2: Limitations are formulaic boilerplate. No bias acknowledgment.
- Score 1: No limitations, biases, or assumptions documented.
OUTPUT FORMAT: Return your assessment as a table followed by a summary:
| Dimension | Score (1-5) | Evidence from Document | Priority Revision |
|---|---|---|---|
| Methodology Justification and Method-Question Fit | |||
| Sampling and Data Source Adequacy | |||
| Triangulation and Multi-Method Strategy | |||
| Data Quality Safeguards | |||
| Transparency, Limitations, and Bias Mitigation |
Total: X/25 Band: Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10) Single Most Important Revision: [One specific sentence]
For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.
DOCUMENT TO SCORE: [Paste your methodology section or full document here]
Fix: Revise What Scored Low
The Revise Prompt
Paste the draft and its scores at the bottom of this prompt. It revises the parts that scored below 4.
You are an expert M&E methodologist. Review the methodology section I will provide and fix every problem that falls below the Good (4) threshold on the Methodology Rigor rubric.
WHAT TO LOOK FOR AND FIX:
Justification problems: Methodology named without justification. No method-question mapping. Alternatives not considered. Context fit assumed.
Sampling problems: Sample size unjustified. Sampling method mismatched to question type. Inclusion criteria implicit. Sample frame undocumented.
Triangulation problems: Single method without acknowledgment. Generic triangulation language. Methods overlap rather than complement.
Quality Safeguard problems: No QA procedures. No consistency checks. No pretesting plan. No data cleaning protocols.
Transparency problems: Generic limitations. Biases unnamed or unmitigated. Assumptions implicit. Replication not feasible.
OUTPUT FORMAT: For each problem found:
- Dimension: [which dimension]
- Location: [which section of the document]
- Problem: [one sentence describing what is wrong]
- Revised version: [the corrected text]
Then provide the complete revised methodology section incorporating all corrections.
DRAFT TO REVISE: [Paste your methodology section here]