Analysis Plan Quality

Last Updated:

On this page

Prepare Your Assessment

Purpose: Score the analysis plan section of any document where one is required, including MEL plans, inception reports, evaluation reports, contribution analyses, and needs assessments. Use to confirm the plan maps to questions, names methods specifically, and includes triangulation and quality checks.

Materials: The protocol or methods section, questions, design and inference purpose, sampling and size rationale, analysis plan and available implementation records.

Stage: State the stage and intended decision. Assess a plan by its documented provisions; completed work requires actual records.

Use: Read the criteria and record located evidence, applicability, ratings and priority revisions. Request missing records. Check AI quotations, source locations, requirements, and scores against the sources; missing evidence does not justify a low score.

Limits: Document quality does not establish readiness, authorization or compliance. Scores remain provisionally checked.

Adaptation and Comparability

Presentation and answer-language changes generally preserve meaning. Changing criteria, anchors, scale or strictness can change the assessment: label it adapted and do not compare its totals with the original rubric. Short mode removes two anchors and requires interpolation. Missing evidence calls for a request for records, not removal of the criterion. These scoring bands are provisionally checked, not practitioner-validated.

Related Reviews and Focused Checks

Start with a full review, then use a focused check for a specific weakness. Do not combine related rubric totals.

AI Prompt Templates

Add your material in the same chat. Context is optional; completed records must come from you.

Assess supplied evidence, report blockers and gaps, and give located findings and priority revisions. Incomplete evidence means no total.

7,546 characters
Adapt this rubric
Establish the document purpose, design, stage, applicable requirements, and available source evidence before rating. Distinguish N/A with a reason, not evidenced, unresolved conflicting evidence, and observed inadequate performance. Missing records do not prove an activity did not occur. Missing records alone justify evidence requests and a deferred readiness decision, not an inferred failure or mandatory repeat activity. Do not invent documentary, signature or approval requirements. Cite document locations for every finding. Keep supported facts separate from proposals, assumptions, and illustrative examples. Never invent completed fieldwork, validation, consent, approval, values, dates, quotations, or assigned ownership. Label unconfirmed owners and proposed revisions for confirmation. Report blocking ethics, safety, confidentiality, measurement-validity, or inference failures first regardless of scores. Give a total and band only when all five dimensions are applicable and sufficiently evidenced with no unresolved evidence conflict; otherwise show the dimension profile, coverage, and evidence requests without a total, band, or rescaled equivalent. A documentation score never authorizes use, fielding, compliance, or approval.

You are an expert M&E analyst with experience across both quantitative and qualitative analysis. Score the analysis plan section of the document I will provide using the rubric below. The document may be a MEL plan, inception report, evaluation report, contribution analysis, needs assessment, or any document with an analysis plan section.

SCORING RUBRIC - Analysis Plan Quality
Rate each applicable, evidenced dimension 1-5. Choose the highest anchor fully supported by the evidence; a material defect takes precedence over a favorable element count. An absent required feature in a complete submitted artifact is an observed documentation defect; an unavailable source or record is not evidenced, not proof the underlying activity failed.

DIMENSION 1: Analysis Plan-to-Question Mapping
- Score 5: All 4 elements: Each research or evaluation question has a defined analytical approach (no orphan questions). The approach is appropriate to the question type (descriptive, inferential, exploratory, explanatory, normative). The order of analytical steps is logical and the sequence is clear. A mapping table or equivalent makes the question-method link visible to a reader.
- Score 4: Three of four elements adequately specified; the remaining element incomplete: Question-method mapping covers most questions; ordering or visible mapping table partial.
- Score 3: Mapping is implicit. Some questions have approaches; others left undefined. Order of analysis loosely described.
- Score 2: Methods listed without clear mapping to questions. Order absent.
- Score 1: No mapping between questions and analysis. Questions and methods sit in unrelated sections.

DIMENSION 2: Analytical Methods Specification
- Score 5: All 4 elements: Specific methods are named (e.g., "logistic regression with the four covariates listed and clustered standard errors at facility level," not "regression analysis"). For qualitative work, the specific approach is named (thematic, framework, grounded theory, content analysis) with a coding strategy described. Software or tools are named where relevant (Stata, R, NVivo, Dedoose, Excel). Method choice is justified (why this method, not another).
- Score 4: Three of four elements adequately specified; the remaining element incomplete: Methods specific and tools named; justification or coding strategy partial.
- Score 3: Methods named at category level only. Tools mentioned. Coding strategy and justification thin.
- Score 2: Generic method labels ("we will analyze qualitatively"). No tools. No justification.
- Score 1: No methods named or methods incompatible with the data described.

DIMENSION 3: Data Preparation and Integration Plan
- Score 5: All 4 elements: Handling of missing data is specified (rules, thresholds, imputation approach if any, or documented exclusion logic). Outlier or anomaly handling is defined with rules for inclusion or exclusion. Data integration logic is clear when combining sources (how survey and admin data merge, how qualitative and quantitative findings connect). Data transformations are documented (recoding, scaling, derived variables).
- Score 4: Three of four elements adequately specified; the remaining element incomplete: Missing data and outliers handled; integration or transformations partial.
- Score 3: Generic statement of "we will handle missingness." Outliers mentioned. Integration vague. Transformations undocumented.
- Score 2: No missingness rules. No outlier rules. Integration assumed.
- Score 1: No data preparation plan.

DIMENSION 4: Triangulation and Synthesis Logic
- Score 5: The evidence strategy fits the questions and justified method or methods. Necessary checks and handling of contradictions are explicit. Integration or weighting is specified where relevant. A single-method analysis can meet this anchor.
- Score 4: The strategy is appropriate and operational; one nonmaterial check or synthesis detail is incomplete.
- Score 3: The strategy is partly described, but important contradictions or the evidence-to-claim link need clarification.
- Score 2: Necessary checks or synthesis are absent, or integration is asserted without an operational method.
- Score 1: The strategy cannot support the claims or falsely presents unsupported checks as validation.

DIMENSION 5: Quality Checks and Validation
- Score 5: Quality checks fit the analytic tradition and risks, with responsibilities and timing. Sensitivity or alternative-interpretation checks are specified where meaningful or nonapplicability explained. Decisions have an audit trail, and limitations state what cannot be answered. Reflexive qualitative work need not use inter-rater reliability.
- Score 4: Appropriate checks, audit trail, and limitations are substantially specified; one minor operational detail remains.
- Score 3: Some appropriate checks are specified but responsibilities, audit trail, or material limitations are incomplete.
- Score 2: Quality assurance is generic or method-inappropriate; material analytical risks lack a response.
- Score 1: No appropriate checking is specified, or validation is claimed beyond what the procedures can establish.

OUTPUT FORMAT:
Return your assessment as a table followed by a summary:

| Dimension | Score (1-5), N/A, or Not Evidenced | Evidence from Document | Priority Revision |
|-----------|-------------|------------------------|-------------------|
| Analysis Plan-to-Question Mapping | | | |
| Analytical Methods Specification | | | |
| Data Preparation and Integration Plan | | | |
| Triangulation and Synthesis Logic | | | |
| Quality Checks and Validation | | | |

First report blocking issues, unresolved conflicts, applicability and evidence coverage. Only if all five dimensions are applicable and sufficiently evidenced with no unresolved source conflict, report **Total: X/25** and **Band:** Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10), as a secondary documentation summary. Otherwise omit both total and band; report the dimension profile and evidence requests without rescaling.
**Single Most Important Revision:** [One specific sentence]

For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.

DOCUMENT TO SCORE:
[Paste your analysis plan section or full document here]
Open it, with the prompt already in:ChatGPTClaude
Or paste it into:GeminiCopilot

Gemini and Copilot open empty. Copy the prompt, then paste it in.

Governing evidence and applicability rules (take precedence over aggregate bands): establish the document purpose, design, stage and available evidence before rating. A fit-for-purpose single-method design can earn the highest applicable rating; do not require complexity, mixed methods or two sources for every finding. Distinguish applicable, not applicable with reason, not evidenced, and observed inadequate. Report applicable coverage and the raw denominator. If any dimension is N/A or not evidenced, report the profile and gaps without a 25-point band or an unvalidated rescaled equivalent. Never interpret missing documentation as proof no activity occurred. Use source-linked historical facts only; keep proposed work and illustrative examples separate. Preserve unresolved source conflicts. Do not invent baseline values, quotations, fieldwork, pilots, validation, consent or approvals to improve a score. A serious consent, confidentiality, safeguarding, measurement-validity or unsupported-inference failure is a blocking issue regardless of the total, including 22/25. High document-quality scores do not establish legal compliance, instrument readiness, field authorization or donor approval. Accountable human review against the applicable protocol is separate. A well-documented failed pilot may score 25/25 for documentation while remaining not ready for fielding.

Use This Rubric

Bring: the study questions, design, data inventory, proposed analysis steps, and method-specific quality procedures.

Stage: State whether this is a proposed design, a draft, or a report of completed work. Assess planned procedures as plans; require records before claiming they occurred. A complete submitted document can have an observed documentation defect even when the underlying activity is unknown.

Limits: Apply requirements to the stated purpose, methods, population, and risk. Do not impose extra methods or sensitive data collection solely to increase a score. These editorial anchors are provisionally checked, not practitioner-calibrated.

Scoring Criteria

Analysis Plan-to-Question Mapping
5Excellent

All 4 elements: Each research or evaluation question has a defined analytical approach (no orphan questions). The approach is appropriate to the question type (descriptive, inferential, exploratory, explanatory, normative). The order of analytical steps is logical and the sequence is clear. A mapping table or equivalent makes the question-method link visible to a reader.

4Good

Three of four elements adequately specified; the remaining element incomplete: Question-method mapping covers most questions; ordering or visible mapping table partial.

3Adequate

Mapping is implicit. Some questions have approaches; others left undefined. Order of analysis loosely described.

2Needs Improvement

Methods listed without clear mapping to questions. Order absent.

1Inadequate

No mapping between questions and analysis. Questions and methods sit in unrelated sections.

Analytical Methods Specification
5Excellent

All 4 elements: Specific methods are named (e.g., "logistic regression with the four covariates listed and clustered standard errors at facility level," not "regression analysis"). For qualitative work, the specific approach is named (thematic, framework, grounded theory, content analysis) with a coding strategy described. Software or tools are named where relevant (Stata, R, NVivo, Dedoose, Excel). Method choice is justified (why this method, not another).

4Good

Three of four elements adequately specified; the remaining element incomplete: Methods specific and tools named; justification or coding strategy partial.

3Adequate

Methods named at category level only. Tools mentioned. Coding strategy and justification thin.

2Needs Improvement

Generic method labels ("we will analyze qualitatively"). No tools. No justification.

1Inadequate

No methods named or methods incompatible with the data described.

Data Preparation and Integration Plan
5Excellent

All 4 elements: Handling of missing data is specified (rules, thresholds, imputation approach if any, or documented exclusion logic). Outlier or anomaly handling is defined with rules for inclusion or exclusion. Data integration logic is clear when combining sources (how survey and admin data merge, how qualitative and quantitative findings connect). Data transformations are documented (recoding, scaling, derived variables).

4Good

Three of four elements adequately specified; the remaining element incomplete: Missing data and outliers handled; integration or transformations partial.

3Adequate

Generic statement of "we will handle missingness." Outliers mentioned. Integration vague. Transformations undocumented.

2Needs Improvement

No missingness rules. No outlier rules. Integration assumed.

1Inadequate

No data preparation plan.

Triangulation and Synthesis Logic
5Excellent

The evidence strategy fits the questions and justified method or methods. Necessary checks and handling of contradictions are explicit. Integration or weighting is specified where relevant. A single-method analysis can meet this anchor.

4Good

The strategy is appropriate and operational; one nonmaterial check or synthesis detail is incomplete.

3Adequate

The strategy is partly described, but important contradictions or the evidence-to-claim link need clarification.

2Needs Improvement

Necessary checks or synthesis are absent, or integration is asserted without an operational method.

1Inadequate

The strategy cannot support the claims or falsely presents unsupported checks as validation.

Quality Checks and Validation
5Excellent

Quality checks fit the analytic tradition and risks, with responsibilities and timing. Sensitivity or alternative-interpretation checks are specified where meaningful or nonapplicability explained. Decisions have an audit trail, and limitations state what cannot be answered. Reflexive qualitative work need not use inter-rater reliability.

4Good

Appropriate checks, audit trail, and limitations are substantially specified; one minor operational detail remains.

3Adequate

Some appropriate checks are specified but responsibilities, audit trail, or material limitations are incomplete.

2Needs Improvement

Quality assurance is generic or method-inappropriate; material analytical risks lack a response.

1Inadequate

No appropriate checking is specified, or validation is claimed beyond what the procedures can establish.

Score Interpretation

Report blockers, the dimension profile, and evidence coverage first. Use these aggregate bands only when all five dimensions are applicable and sufficiently evidenced with no unresolved evidence conflict. Otherwise give no total, band, or rescaled equivalent. High documentation quality does not establish readiness, legal compliance, or approval.

Total (out of 25)BandNext Step
22-25StrongStrong documentation profile only. Resolve blocking issues and obtain the applicable human review; this score does not authorize use.
17-21AdequatePrioritize source-linked revisions in the weakest dimensions. Request missing records before proposing additional activity; confirm the need for any new work from evidence.
11-16Needs RevisionUse located findings as a revision brief. Distinguish reporting gaps from demonstrated design or implementation failures and label proposed work for confirmation.
5-10Substantial RevisionRework the evidence-backed weaknesses with accountable review. A low documentation profile alone does not establish that the underlying activity failed or must be repeated.
Did this page help?