You are an expert M&E data collection specialist. Score the instrument pilot report I have attached or pasted using the rubric below.
SCORING RUBRIC - Instrument Pilot Results
Score each dimension 1-5 using these criteria:
DIMENSION 1: Pilot Scope Documentation
| 5 | All four elements present. Participant count stated, demographic breakdown documented (sex, age, language, or other relevant strata for the instrument), selection method named (purposive criteria, convenience, or other rationale), and pilot location and timing documented (where and when the pilot took place). |
|---|---|
| 4 | At least three of four elements present. Count and demographics documented; selection method or timing partial. |
| 3 | At least two of four elements present. Count stated and one other element. |
| 2 | Pilot participant count stated without demographics or selection rationale. |
| 1 | No pilot scope documented, OR pilot scope so limited it cannot test the instrument (e.g., one participant for a household survey). |
DIMENSION 2: Issues Identified
| 5 | All four elements present. Comprehension issues cataloged (specific items, questions, or terms that participants misunderstood), length and fatigue issues noted with quantitative reference where possible (e.g., "interview duration averaged 78 minutes against 45-minute target"), sensitive content issues noted (questions that prompted refusal, distress, or non-response), and skip-logic or instrument-flow failures documented. |
|---|---|
| 4 | At least three of four elements present. Multiple issue types cataloged; one issue type partial. |
| 3 | At least two of four elements present. Some issues documented but generic. |
| 2 | Issues referenced without specificity (e.g., "some questions were confusing"). |
| 1 | No issues identified, OR pilot report states no issues despite obvious instrument problems. |
DIMENSION 3: Revisions Documented
| 5 | All four elements present. Each identified issue paired with a specific revision (revised wording, added or removed item, modified skip logic, restructured section), rationale for each revision provided, items intentionally not revised noted with rationale (e.g., "kept original wording for cross-period comparability"), and final revised instrument is referenced or attached. |
|---|---|
| 4 | At least three of four elements present. Issues paired with revisions; rationale or unrevised items partial. |
| 3 | At least two of four elements present. Major revisions documented but minor revisions or rationale absent. |
| 2 | Revisions referenced generically ("instrument was updated") without item-level specificity. |
| 1 | No revisions documented, OR revisions not tied to identified issues. |
DIMENSION 4: Translator and Enumerator Feedback
| 5 | All four elements present. Enumerator feedback collected and summarized (what was hard to administer, what felt awkward), translator feedback collected where translation is involved (terms that did not translate cleanly, cultural appropriateness issues), specific examples of feedback included (not just a summary), and feedback mechanism documented (debrief session, structured form, written notes). |
|---|---|
| 4 | At least three of four elements present. Enumerator and translator feedback collected; examples or mechanism partial. |
| 3 | At least two of four elements present. Enumerator feedback included but translator feedback absent (where translation applies), or vice versa. |
| 2 | Administration-side feedback referenced without examples or mechanism. |
| 1 | No enumerator or translator feedback. |
DIMENSION 5: Field-Readiness Statement
| 5 | All four elements present. Explicit field-readiness statement included (instrument is ready, instrument is conditionally ready, or instrument requires further revision), remaining issues listed if any with planned resolution, decision-maker for field-readiness named (e.g., research lead, MEL manager), and any further testing planned before fielding documented. |
|---|---|
| 4 | At least three of four elements present. Field-readiness stated with remaining issues; decision-maker or further testing partial. |
| 3 | At least two of four elements present. Readiness stated but remaining issues vague. |
| 2 | Pilot report ends with revisions but no explicit readiness assessment. |
| 1 | No field-readiness statement, OR pilot results not used to inform a launch decision. |
OUTPUT FORMAT:
Return your assessment as a table followed by a summary:
| Dimension | Score (1-5) | Evidence from Pilot Report | Priority Revision |
|---|---|---|---|
| Pilot Scope Documentation | |||
| Issues Identified | |||
| Revisions Documented | |||
| Translator and Enumerator Feedback | |||
| Field-Readiness Statement |
Total: X/25 Band: Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10) Single Most Important Revision: [One specific sentence] Field-Readiness Recommendation: [Ready to field / Field with stated conditions / Repilot before fielding / Major instrument revision needed]
For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.
INSTRUMENT PILOT REPORT TO SCORE:
(See the attached or pasted document.)
Document to review: Attached, or pasted below. Base everything above on it and do not invent details that are not in it.