Design an Instrument Pilot Test
Design a pilot test for a survey, KII guide, FGD guide, or observation checklist, including participant scope, issue catalog, revision protocol, and field-readiness criteria.
Draft: Design an Instrument Pilot Test
Your Prompt
- What You Need
- Share the actual instrument and configuration, language versions, population, consent and safeguarding protocols, resources and approval route.
- Choose This Prompt When
- Use this to plan empirical testing of an instrument. Choose Review a Survey Instrument for a desk review, or Create an Enumerator Training Plan to prepare the collection team.
Describe your goal, audience, available information and constraints. Your words are added to the prompt as supplied. You can also use the guided fields below.
Kept in this tab while you work. Copy and Start includes this description.
Copy this prompt and paste it into your AI chat to start. If it needs source material, the AI will ask for it.
Evidence and applicability: use only supplied or verifiable evidence for historical facts, values, quotations, completed activities and approvals. Preserve conflicting sources with their provenance; do not choose a preferred value without adjudication. Mark missing evidence as unknown or not evidenced, not as proof an activity never occurred. Keep proposed plans and illustrative examples separate from findings. Never invent a baseline, quote, pilot, validation or sign-off to complete a template. Match methods to the purpose and population; a suitable single-method design is valid. Distinguish descriptive change from causal attribution. Check obligations in the applicable funding agreement rather than assuming a universal donor rule.
You are an expert M&E data collection specialist designing a pilot test for your instrument and target population.
Your pilot test design must adhere to the following requirements:
- Pilot scope. Propose a scope, relevant participant diversity, selection rationale, location and timing matched to the instrument's risks, safety and available resources; justify any count and mark unresolved inputs. A diagnostic pilot need not be statistically representative. State why the scope is adequate to surface the issues the instrument is most at risk of carrying into the field.
- Issue catalog. Define the categories of issues the pilot will systematically watch for: comprehension issues (specific items, questions, or terms that participants misunderstand), length and fatigue (with consented timing and observation appropriate to the instrument), sensitive content (questions that prompt refusal, distress, or non-response), and skip-logic or instrument-flow failures.
- Revision protocol. Specify how issues identified in the pilot will be translated into instrument revisions: who reviews the pilot notes, who has authority to change items, how revisions are version-controlled, and how the team will avoid silent unrecorded fixes.
- Translator and enumerator feedback. Build in structured feedback from those who administered the instrument: a short debrief protocol covering items that were hard to ask, items that produced inconsistent answers, and items that translated poorly across languages.
- Field-readiness criteria. State the criteria the instrument must meet before launch and the format of the field-readiness statement (or the explicit list of remaining issues to resolve before launch).
Output Format:
Produce:
- A short framing paragraph stating the pilot's purpose and proposed criteria for the responsible owner's later field-readiness decision.
- A pilot scope table (count, demographics, selection, location, timing).
- An issue-tracking template (issue category, observation, affected items, severity, decision, revision made).
- A focused debrief protocol for translators and enumerators.
- A proposed field-readiness checklist, unresolved issues and sign-off route. Do not issue a go/no-go decision before the pilot evidence and authorized review exist.
Inputs and ethics: Request the actual instrument, instructions, routing or configured form, language versions, population, study purpose, sampling and consent procedures, sensitive-topic safeguards, resources and approval route. A tool name or platform is not its content. Separate cognitive testing, technical checks and field rehearsal as needed. Plan voluntary consent/assent and any required ethics review, confidentiality, safe referral and data handling; avoid collecting unnecessary identifiers. Retest material revisions. This is a pilot design and issue-log template, not a claim that testing or clearance occurred.
Check: Score It with the Instrument Pilot Results Rubric
The Review Prompt
Paste the draft at the bottom of this prompt and run it. It scores each criterion from 1 to 5 and says why.
Establish the document purpose, design, stage, applicable requirements, and available source evidence before rating. Distinguish N/A with a reason, not evidenced, unresolved conflicting evidence, and observed inadequate performance. Missing records do not prove an activity did not occur. Missing records alone justify evidence requests and a deferred readiness decision, not an inferred failure or mandatory repeat activity. Do not invent documentary, signature or approval requirements. Cite document locations for every finding. Keep supported facts separate from proposals, assumptions, and illustrative examples. Never invent completed fieldwork, validation, consent, approval, values, dates, quotations, or assigned ownership. Label unconfirmed owners and proposed revisions for confirmation. Report blocking ethics, safety, confidentiality, measurement-validity, or inference failures first regardless of scores. Give a total and band only when all five dimensions are applicable and sufficiently evidenced with no unresolved evidence conflict; otherwise show the dimension profile, coverage, and evidence requests without a total, band, or rescaled equivalent. A documentation score never authorizes use, fielding, compliance, or approval.
You are an expert M&E data collection specialist. Score the instrument pilot report I will provide using the rubric below.
SCORING RUBRIC - Instrument Pilot Results Rate each applicable, evidenced dimension 1-5. Choose the highest anchor fully supported by the evidence; a material defect takes precedence over a favorable element count. An absent required feature in a complete submitted artifact is an observed documentation defect; an unavailable source or record is not evidenced, not proof the underlying activity failed.
DIMENSION 1: Pilot Scope Documentation
- Score 5: Actual pilot scope is documented: participant or case count, relevant characteristics, selection rationale, version, location, and timing. Adequacy is justified against what the pilot aimed to learn, without a universal minimum size.
- Score 4: Actual scope and version are clear; a minor characteristic, selection, or timing detail is incomplete.
- Score 3: Count and some scope details are documented but important context or adequacy rationale is unclear.
- Score 2: Pilot scope is described generically without enough detail to judge intended checks.
- Score 1: No usable account of scope is supplied or documented scope cannot address its stated purpose. Missing records remain not evidenced.
DIMENSION 2: Issues Identified
- Score 5: The report documents which comprehension, burden, sensitivity, and routing checks occurred, what was found, and the item or version. A supported finding of no detected issue is allowed. Untested areas and limits are explicit.
- Score 4: Checks and principal issues or no-issue observations are traceable; a minor item or burden detail remains.
- Score 3: Some checks and findings are documented but material item, scope, or measurement details are incomplete.
- Score 2: Issue or no-issue claims are generic and not traceable to actual checks.
- Score 1: Pilot findings contradict records or known material defects are concealed. No detected issue alone is not a documentation failure.
DIMENSION 3: Revisions Documented
- Score 5: Every identified issue has an evidenced revision, a proposed revision with status, or a justified no-change decision. Rationale and unresolved issues are explicit. The actual tested or current revised draft version is referenced with status; a final or field-ready instrument is not required for excellent documentation. A justified no-change pilot can meet this anchor.
- Score 4: Issue-to-action or justified no-change traceability and version status are clear; one minor rationale or reference detail is incomplete.
- Score 3: Major issues and decisions are documented but material status, rationale, or version details need clarification.
- Score 2: Changes or no-change decisions are described generically without usable issue-level traceability.
- Score 1: The account conceals known changes or unresolved issues, contradicts version evidence, or falsely claims corrections were completed. No revision alone is not a failure when the no-change decision is justified.
DIMENSION 4: Translator and Enumerator Feedback
- Score 5: Applicable administration-side feedback is evidenced and summarized with specific examples and the collection mechanism. Include enumerator feedback when enumerators administered the instrument, translator feedback when translation was involved, or relevant self-administration observations. Explain nonapplicability; no fixed four-element denominator applies.
- Score 4: Applicable feedback and mechanism are substantially documented; a minor example or context detail is incomplete.
- Score 3: Some applicable feedback is documented but material examples, coverage, or mechanism details are unclear.
- Score 2: Administration-side feedback is asserted generically without traceable examples or process.
- Score 1: The report misrepresents feedback or suppresses an evidenced administration problem. Unavailable records remain not evidenced.
DIMENSION 5: Field-Readiness Statement
- Score 5: The actual readiness decision and its source are explicit, or the decision is transparently deferred pending evidence. Known unresolved issues and their status are listed; the accountable human decision function is identified from records or marked unconfirmed. Any further testing is clearly distinguished as completed, planned, or conditionally proposed. Documentation quality does not override a failed pilot.
- Score 4: Decision or evidence-based deferral and known issues are clear; one nonmaterial source or follow-up detail remains.
- Score 3: Some decision or follow-up information is present but its source, status, or relation to unresolved issues needs material clarification.
- Score 2: The report implies readiness without addressing evidenced material defects or blurs completed and proposed actions.
- Score 1: The report claims unsupported field authorization or suppresses an evidenced blocking defect. Missing records alone require an evidence request and deferred decision, not inferred failure or mandatory repetition.
OUTPUT FORMAT: Return your assessment as a table followed by a summary:
| Dimension | Score (1-5), N/A, or Not Evidenced | Evidence from Pilot Report | Priority Revision |
|---|---|---|---|
| Pilot Scope Documentation | |||
| Issues Identified | |||
| Revisions Documented | |||
| Translator and Enumerator Feedback | |||
| Field-Readiness Statement |
First report blocking issues, unresolved conflicts, applicability and evidence coverage. Only if all five dimensions are applicable and sufficiently evidenced with no unresolved source conflict, report Total: X/25 and Band: Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10), as a secondary documentation summary. Otherwise omit both total and band; report the dimension profile and evidence requests without rescaling. Single Most Important Revision: [One specific sentence] Reported Readiness Decision and Source: State only the decision recorded in supplied evidence. If it is unavailable, write not evidenced; defer the readiness decision pending records. Missing records alone do not require repiloting, signatures, named annexes, or repeat activities. Do not infer timing, location, participant characteristics, or completed changes from surrounding context. Proposed Next Actions: Evidence requests first. Describe any further testing as a conditional proposal linked to an evidenced unresolved defect, never as completed work or an invented requirement.
For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.
INSTRUMENT PILOT REPORT TO SCORE: [Paste your instrument pilot report here]
Fix: Revise What Scored Low
The Revise Prompt
Paste the draft and its scores at the bottom of this prompt. It revises the parts that scored below 4.
Establish the document purpose, design, stage, applicable requirements, and available source evidence before rating. Distinguish N/A with a reason, not evidenced, unresolved conflicting evidence, and observed inadequate performance. Missing records do not prove an activity did not occur. Missing records alone justify evidence requests and a deferred readiness decision, not an inferred failure or mandatory repeat activity. Do not invent documentary, signature or approval requirements. Cite document locations for every finding. Keep supported facts separate from proposals, assumptions, and illustrative examples. Never invent completed fieldwork, validation, consent, approval, values, dates, quotations, or assigned ownership. Label unconfirmed owners and proposed revisions for confirmation. Report blocking ethics, safety, confidentiality, measurement-validity, or inference failures first regardless of scores. Give a total and band only when all five dimensions are applicable and sufficiently evidenced with no unresolved evidence conflict; otherwise show the dimension profile, coverage, and evidence requests without a total, band, or rescaled equivalent. A documentation score never authorizes use, fielding, compliance, or approval.
You are an expert M&E data collection specialist. Review the draft instrument pilot report I will provide and fix every problem that falls below the Good (4) threshold on the Instrument Pilot Results rubric.
Separate supported corrections from evidence requests and proposed text requiring confirmation. A plausible owner, date, rationale, or account of an activity is not a verified fact. Preserve unresolved source conflicts.
WHAT TO LOOK FOR AND FIX:
Pilot Scope problems: Participant or case count stated without relevant characteristics. No selection method or rationale. Pilot location or timing unspecified.
Issues problems: Issues referenced generically ("some questions were confusing"). No quantitative reference for length. Sensitive content issues not noted. Skip-logic failures not documented.
Revisions problems: Revisions referenced as "instrument updated" without item-level specificity. No rationale per revision. Items intentionally not revised not noted. Current tested or revised version and its status not referenced.
Feedback problems: Applicable administration-side feedback is missing: enumerator feedback when enumerators administered the instrument, translator feedback where translation applies, or relevant self-administration observations. No specific examples. Feedback mechanism not documented. Explain nonapplicability; unavailable records remain not evidenced.
Field-Readiness problems: No explicit readiness statement. Remaining issues vague. No decision-maker named. Further testing not addressed.
OUTPUT FORMAT: For each problem found:
- Dimension: [which dimension]
- Location: [which section of the pilot report]
- Problem: [one sentence describing what is wrong]
- Revised version: [the corrected text]
Then provide the complete revised pilot report incorporating all corrections.
DRAFT INSTRUMENT PILOT REPORT TO REVISE: [Paste your draft instrument pilot report here]