You are an expert M&E methodologist with experience across both quantitative and qualitative methods. Score the methodology section of the document I have attached or pasted using the rubric below. The document may be a MEL plan, evaluation ToR, inception report, evaluation report, contribution analysis, needs assessment, sampling plan, or any document where methodology choices need justification and quality safeguards.
SCORING RUBRIC - Methodology Rigor
Score each dimension 1-5 using these criteria:
DIMENSION 1: Methodology Justification and Method-Question Fit
| 5 | All four elements present. Methodology is explicitly justified relative to the research questions or evaluation questions (not just named). Each major method maps to specific questions it is designed to answer. Alternatives have been considered and rejected with documented reasoning. The choice is appropriate to the context (operating environment, data availability, ethical constraints, capacity), not just academic best practice in the abstract. |
|---|---|
| 4 | At least three of four elements present. Methodology justified and mapped to questions; alternatives or context considerations partial. |
| 3 | Methodology named and broadly tied to questions but justification is thin. Alternatives not considered. Context fit assumed rather than examined. |
| 2 | Methodology named without justification. No method-question mapping. |
| 1 | No methodology described or methodology disconnected from research questions. |
DIMENSION 2: Sampling and Data Source Adequacy
| 5 | All four elements present. Sample size or data source coverage is justified (statistical power calculation for quantitative, saturation logic for qualitative, comprehensive coverage logic for document review). Sampling method is appropriate to the question type (purposive for explanatory, probability for prevalence, etc.). Inclusion and exclusion criteria are clear and operational. Sample frame is documented (where the sample is drawn from, how it is constructed). |
|---|---|
| 4 | At least three of four elements present. Sample size justified and method appropriate; inclusion criteria or sample frame partial. |
| 3 | Sampling described but justification weak. Inclusion criteria implicit. Sample frame partially documented. |
| 2 | Sampling described with minimal justification. Coverage adequacy unclear. |
| 1 | No sampling rationale or sample frame. |
DIMENSION 3: Triangulation and Multi-Method Strategy
| 5 | All four elements present. Multiple data sources or methods are used for key research questions (not just for the overall study). Triangulation strategy is explicit (which finding will be cross-checked with which source, when, how). Different methods address different aspects of the question (e.g., quantitative for prevalence, qualitative for mechanism). Sources or methods complement each other rather than duplicating coverage. |
|---|---|
| 4 | At least three of four elements present. Multiple methods used; triangulation strategy partial or some methods overlap. |
| 3 | Multiple methods present but triangulation logic vague ("we will triangulate"). Methods overlap on coverage. |
| 2 | Single method or two methods used redundantly. No triangulation strategy. |
| 1 | Single method without acknowledgment of its limitations or need for triangulation. |
DIMENSION 4: Data Quality Safeguards
| 5 | All four elements present. Quality assurance procedures documented (supervision, spot checks, back-translation if relevant, audit trails). Consistency checks specified where relevant (inter-rater reliability for coding, double entry for surveys). Pretesting or piloting is planned and described for any primary data collection. Data cleaning protocols specified (rules for handling outliers, missing data, contradictory responses). |
|---|---|
| 4 | At least three of four elements present. QA procedures and pretesting documented; consistency checks or cleaning partial. |
| 3 | Some QA procedures named but not operationalized. Pretesting mentioned but not described. Cleaning protocols absent. |
| 2 | Generic mention of "quality" without specific procedures. No pretesting plan. |
| 1 | No data quality safeguards described. |
DIMENSION 5: Transparency, Limitations, and Bias Mitigation
| 5 | All four elements present. Limitations are explicitly stated and specific (not generic "our study has limitations"). Biases are named with concrete mitigation strategies (e.g., "selection bias from voluntary participation will be mitigated by also interviewing non-participants"). Assumptions are documented (what we assumed to be true that affects findings). Replication is feasible: another researcher could reproduce key findings from what is documented. |
|---|---|
| 4 | At least three of four elements present. Limitations and biases named; mitigation or replicability partial. |
| 3 | Generic limitations stated. Biases mentioned without mitigation. Assumptions implicit. |
| 2 | Limitations are formulaic boilerplate. No bias acknowledgment. |
| 1 | No limitations, biases, or assumptions documented. |
OUTPUT FORMAT:
Return your assessment as a table followed by a summary:
| Dimension | Score (1-5) | Evidence from Document | Priority Revision |
|---|---|---|---|
| Methodology Justification and Method-Question Fit | |||
| Sampling and Data Source Adequacy | |||
| Triangulation and Multi-Method Strategy | |||
| Data Quality Safeguards | |||
| Transparency, Limitations, and Bias Mitigation |
Total: X/25 Band: Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10) Single Most Important Revision: [One specific sentence]
For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.
DOCUMENT TO SCORE:
(See the attached or pasted document.)
Document to review: Attached, or pasted below. Base everything above on it and do not invent details that are not in it.