Design an Instrument Pilot Test
Design a pilot test for a survey, KII guide, FGD guide, or observation checklist, including participant scope, issue catalog, revision protocol, and field-readiness criteria.
Draft: Design an Instrument Pilot Test
Fill In Your Details (3 Blanks)
Add your information to get a tailored prompt. You can also use it as is.
Sector and Donor Context (Optional)
Your Prompt (With Your Details)
This is the complete prompt, ready to copy and paste.
You are an expert M&E data collection specialist designing a pilot test for your instrument and target population.
Your pilot test design must adhere to the following requirements:
- Pilot scope. Specify participant count, demographic breakdown (sex, age, language, or other relevant strata for the instrument), selection method (purposive criteria, convenience, or other rationale), and pilot location and timing. State why the scope is adequate to surface the issues the instrument is most at risk of carrying into the field.
- Issue catalog. Define the categories of issues the pilot will systematically watch for: comprehension issues (specific items, questions, or terms that participants misunderstand), length and fatigue (with stopwatch-based timing), sensitive content (questions that prompt refusal, distress, or non-response), and skip-logic or instrument-flow failures.
- Revision protocol. Specify how issues identified in the pilot will be translated into instrument revisions: who reviews the pilot notes, who has authority to change items, how revisions are version-controlled, and how the team will avoid silent unrecorded fixes.
- Translator and enumerator feedback. Build in structured feedback from those who administered the instrument: a short debrief protocol covering items that were hard to ask, items that produced inconsistent answers, and items that translated poorly across languages.
- Field-readiness criteria. State the criteria the instrument must meet before launch and the format of the field-readiness statement (or the explicit list of remaining issues to resolve before launch).
Output Format:
Produce:
- A short framing paragraph stating the pilot's purpose and what "good enough to field" looks like.
- A pilot scope table (count, demographics, selection, location, timing).
- An issue-tracking template (issue category, observation, affected items, severity, decision, revision made).
- A debrief protocol (one short section, 5-8 questions) for translators and enumerators.
- A field-readiness criteria checklist with a clear go/no-go statement.
Check: Score It with the Instrument Pilot Results Rubric
The Review Prompt
Paste the draft at the bottom of this prompt and run it. It scores each criterion from 1 to 5 and says why.
You are an expert M&E data collection specialist. Score the instrument pilot report I will provide using the rubric below.
SCORING RUBRIC - Instrument Pilot Results Score each dimension 1-5 using these criteria:
DIMENSION 1: Pilot Scope Documentation
- Score 5: All four elements present. Participant count stated, demographic breakdown documented (sex, age, language, or other relevant strata for the instrument), selection method named (purposive criteria, convenience, or other rationale), and pilot location and timing documented (where and when the pilot took place).
- Score 4: At least three of four elements present. Count and demographics documented; selection method or timing partial.
- Score 3: At least two of four elements present. Count stated and one other element.
- Score 2: Pilot participant count stated without demographics or selection rationale.
- Score 1: No pilot scope documented, OR pilot scope so limited it cannot test the instrument (e.g., one participant for a household survey).
DIMENSION 2: Issues Identified
- Score 5: All four elements present. Comprehension issues cataloged (specific items, questions, or terms that participants misunderstood), length and fatigue issues noted with quantitative reference where possible (e.g., "interview duration averaged 78 minutes against 45-minute target"), sensitive content issues noted (questions that prompted refusal, distress, or non-response), and skip-logic or instrument-flow failures documented.
- Score 4: At least three of four elements present. Multiple issue types cataloged; one issue type partial.
- Score 3: At least two of four elements present. Some issues documented but generic.
- Score 2: Issues referenced without specificity (e.g., "some questions were confusing").
- Score 1: No issues identified, OR pilot report states no issues despite obvious instrument problems.
DIMENSION 3: Revisions Documented
- Score 5: All four elements present. Each identified issue paired with a specific revision (revised wording, added or removed item, modified skip logic, restructured section), rationale for each revision provided, items intentionally not revised noted with rationale (e.g., "kept original wording for cross-period comparability"), and final revised instrument is referenced or attached.
- Score 4: At least three of four elements present. Issues paired with revisions; rationale or unrevised items partial.
- Score 3: At least two of four elements present. Major revisions documented but minor revisions or rationale absent.
- Score 2: Revisions referenced generically ("instrument was updated") without item-level specificity.
- Score 1: No revisions documented, OR revisions not tied to identified issues.
DIMENSION 4: Translator and Enumerator Feedback
- Score 5: All four elements present. Enumerator feedback collected and summarized (what was hard to administer, what felt awkward), translator feedback collected where translation is involved (terms that did not translate cleanly, cultural appropriateness issues), specific examples of feedback included (not just a summary), and feedback mechanism documented (debrief session, structured form, written notes).
- Score 4: At least three of four elements present. Enumerator and translator feedback collected; examples or mechanism partial.
- Score 3: At least two of four elements present. Enumerator feedback included but translator feedback absent (where translation applies), or vice versa.
- Score 2: Administration-side feedback referenced without examples or mechanism.
- Score 1: No enumerator or translator feedback.
DIMENSION 5: Field-Readiness Statement
- Score 5: All four elements present. Explicit field-readiness statement included (instrument is ready, instrument is conditionally ready, or instrument requires further revision), remaining issues listed if any with planned resolution, decision-maker for field-readiness named (e.g., research lead, MEL manager), and any further testing planned before fielding documented.
- Score 4: At least three of four elements present. Field-readiness stated with remaining issues; decision-maker or further testing partial.
- Score 3: At least two of four elements present. Readiness stated but remaining issues vague.
- Score 2: Pilot report ends with revisions but no explicit readiness assessment.
- Score 1: No field-readiness statement, OR pilot results not used to inform a launch decision.
OUTPUT FORMAT: Return your assessment as a table followed by a summary:
| Dimension | Score (1-5) | Evidence from Pilot Report | Priority Revision |
|---|---|---|---|
| Pilot Scope Documentation | |||
| Issues Identified | |||
| Revisions Documented | |||
| Translator and Enumerator Feedback | |||
| Field-Readiness Statement |
Total: X/25 Band: Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10) Single Most Important Revision: [One specific sentence] Field-Readiness Recommendation: [Ready to field / Field with stated conditions / Repilot before fielding / Major instrument revision needed]
For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.
INSTRUMENT PILOT REPORT TO SCORE: [Paste your instrument pilot report here]
Fix: Revise What Scored Low
The Revise Prompt
Paste the draft and its scores at the bottom of this prompt. It revises the parts that scored below 4.
You are an expert M&E data collection specialist. Review the draft instrument pilot report I will provide and fix every problem that falls below the Good (4) threshold on the Instrument Pilot Results rubric.
WHAT TO LOOK FOR AND FIX:
Pilot Scope problems: Participant count stated without demographics. No selection method or rationale. Pilot location or timing unspecified.
Issues problems: Issues referenced generically ("some questions were confusing"). No quantitative reference for length. Sensitive content issues not noted. Skip-logic failures not documented.
Revisions problems: Revisions referenced as "instrument updated" without item-level specificity. No rationale per revision. Items intentionally not revised not noted. Final instrument not referenced.
Feedback problems: No enumerator feedback. No translator feedback (where translation applies). No specific examples. Feedback mechanism not documented.
Field-Readiness problems: No explicit readiness statement. Remaining issues vague. No decision-maker named. Further testing not addressed.
OUTPUT FORMAT: For each problem found:
- Dimension: [which dimension]
- Location: [which section of the pilot report]
- Problem: [one sentence describing what is wrong]
- Revised version: [the corrected text]
Then provide the complete revised pilot report incorporating all corrections.
DRAFT INSTRUMENT PILOT REPORT TO REVISE: [Paste your draft instrument pilot report here]