You are an expert M&E methodologist with experience reviewing evaluation reports for donors and program teams. Score the limitations section of the report I have attached or pasted using the rubric below.
SCORING RUBRIC - Limitations Disclosure
Score each dimension 1-5 using these criteria:
DIMENSION 1: Sampling Limitations
| 5 | All elements present. Sampling-related limitations are named specifically: coverage gaps (groups, geographies, time periods missed), response patterns (non-response, attrition, selection into the sample), selection bias risks, and exclusions made during fielding (sites dropped, groups deferred). Specific numbers given where relevant. |
|---|---|
| 4 | At least three of four elements present. Most sampling issues named; one element partial or generic. |
| 3 | At least two of four elements present. Sampling limitations named but vague. |
| 2 | Generic statement that "sampling was not random" or similar, without specifics. |
| 1 | Absent or inadequate. No sampling limitations disclosed despite obvious gaps. |
DIMENSION 2: Method-Specific Limitations
| 5 | All elements present. Each method used has its limitations stated specifically. Survey limitations (response rates, instrument issues), KII or FGD limitations (depth, group dynamics, facilitator effects), document review limitations (availability, currency, selection), observation limitations (Hawthorne effects, coverage). Limitations match the methods actually used. |
|---|---|
| 4 | At least three of four elements present. Most methods covered; one method treated generically. |
| 3 | At least two of four elements present. Some methods covered with specifics; others collapsed into generic statements. |
| 2 | Limitations stated as a single generic paragraph that does not distinguish methods. |
| 1 | Absent or inadequate. No method-specific limitations. |
DIMENSION 3: Data Quality Limitations
| 5 | All elements present. Missing data patterns disclosed with counts or percentages where relevant. Recall problems acknowledged for retrospective questions. Social desirability bias acknowledged where present. Translation or interpretation issues disclosed. Inconsistencies found during cleaning are named. |
|---|---|
| 4 | At least three of four elements present. Most data quality issues disclosed; one element partial. |
| 3 | At least two of four elements present. Some data quality issues disclosed; others omitted. |
| 2 | Data quality stated as "generally good" or similar, without specifics. |
| 1 | Absent or inadequate. No data quality limitations disclosed. |
DIMENSION 4: Impact on Findings
| 5 | All elements present. Each limitation is tied to specific findings or conclusions it affects. The direction of bias is named where it can be inferred. Findings that should be interpreted with extra caution are flagged. Findings unaffected by the limitations are also noted. |
|---|---|
| 4 | At least three of four elements present. Most limitations tied to findings; one element partial. |
| 3 | At least two of four elements present. Some limitations tied to findings; others listed separately. |
| 2 | Limitations listed as a standalone section with no link to specific findings. |
| 1 | Absent or inadequate. No impact-on-findings analysis. |
DIMENSION 5: Mitigation Statement
| 5 | All elements present. Each limitation has a stated mitigation (what was done during design, fielding, or analysis to reduce its effect). The residual uncertainty after mitigation is named. Where mitigation was not possible, this is stated honestly rather than glossed over. Mitigations are concrete (e.g., "triangulated survey results with KIIs from the same sites") rather than aspirational. |
|---|---|
| 4 | At least three of four elements present. Most limitations have mitigations; one element partial. |
| 3 | At least two of four elements present. Some mitigations stated; others left as bare limitations. |
| 2 | Mitigations stated as generic intentions ("we tried to minimize this") without specifics. |
| 1 | Absent or inadequate. No mitigation statements. |
OUTPUT FORMAT:
Return your assessment as a table followed by a summary:
| Dimension | Score (1-5) | Evidence | Priority Revision |
|---|---|---|---|
| Sampling Limitations | |||
| Method-Specific Limitations | |||
| Data Quality Limitations | |||
| Impact on Findings | |||
| Mitigation Statement |
Total: X/25 Band: Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10) Single Most Important Revision: [One specific sentence]
For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.
LIMITATIONS SECTION TO SCORE:
(See the attached or pasted document.)
Document to review: Attached, or pasted below. Base everything above on it and do not invent details that are not in it.