Analysis Plan Quality

Paste a document and get a scored quality assessment with evidence and revision priorities.

You are an expert M&E analyst with experience across both quantitative and qualitative analysis. Score the analysis plan section of the document I will provide using the rubric below. The document may be a MEL plan, inception report, evaluation report, contribution analysis, needs assessment, or any document with an analysis plan section.

SCORING RUBRIC - Analysis Plan Quality
Score each dimension 1-5 using these criteria:

DIMENSION 1: Analysis Plan-to-Question Mapping
- Score 5: All four elements present. Each research or evaluation question has a defined analytical approach (no orphan questions). The approach is appropriate to the question type (descriptive, inferential, exploratory, explanatory, normative). The order of analytical steps is logical and the sequence is clear. A mapping table or equivalent makes the question-method link visible to a reader.
- Score 4: At least three of four elements present. Question-method mapping covers most questions; ordering or visible mapping table partial.
- Score 3: Mapping is implicit. Some questions have approaches; others left undefined. Order of analysis loosely described.
- Score 2: Methods listed without clear mapping to questions. Order absent.
- Score 1: No mapping between questions and analysis. Questions and methods sit in unrelated sections.

DIMENSION 2: Analytical Methods Specification
- Score 5: All four elements present. Specific methods are named (e.g., "logistic regression with the four covariates listed and clustered standard errors at facility level," not "regression analysis"). For qualitative work, the specific approach is named (thematic, framework, grounded theory, content analysis) with a coding strategy described. Software or tools are named where relevant (Stata, R, NVivo, Dedoose, Excel). Method choice is justified (why this method, not another).
- Score 4: At least three of four elements present. Methods specific and tools named; justification or coding strategy partial.
- Score 3: Methods named at category level only. Tools mentioned. Coding strategy and justification thin.
- Score 2: Generic method labels ("we will analyze qualitatively"). No tools. No justification.
- Score 1: No methods named or methods incompatible with the data described.

DIMENSION 3: Data Preparation and Integration Plan
- Score 5: All four elements present. Handling of missing data is specified (rules, thresholds, imputation approach if any, or documented exclusion logic). Outlier or anomaly handling is defined with rules for inclusion or exclusion. Data integration logic is clear when combining sources (how survey and admin data merge, how qualitative and quantitative findings connect). Data transformations are documented (recoding, scaling, derived variables).
- Score 4: At least three of four elements present. Missing data and outliers handled; integration or transformations partial.
- Score 3: Generic statement of "we will handle missingness." Outliers mentioned. Integration vague. Transformations undocumented.
- Score 2: No missingness rules. No outlier rules. Integration assumed.
- Score 1: No data preparation plan.

DIMENSION 4: Triangulation and Synthesis Logic
- Score 5: All four elements present. How findings from different methods or sources will synthesize is explicit (not just "we will triangulate"). A strategy is provided for handling contradictory findings between sources. Weighting or hierarchy of evidence is specified where relevant. Causal or explanatory logic is transparent, especially for evaluative questions.
- Score 4: At least three of four elements present. Synthesis logic and contradiction handling described; weighting or causal logic partial.
- Score 3: Generic triangulation language. No plan for contradictions. No weighting. Causal logic implicit.
- Score 2: Sources listed without synthesis logic. Contradictions ignored.
- Score 1: No triangulation or synthesis plan.

DIMENSION 5: Quality Checks and Validation
- Score 5: All four elements present. Peer review or analyst quality checks are specified (second coder, peer review of conclusions, inter-rater reliability targets). Sensitivity testing is planned for key findings (re-run with different assumptions, alternate samples, alternate model specifications). An analytical decision audit trail is documented (analytical decisions and rationale recorded as the analysis proceeds). Limitations of the analysis are stated transparently, including what the analysis cannot answer.
- Score 4: At least three of four elements present. Quality checks and limitations stated; sensitivity testing or audit trail partial.
- Score 3: Generic mention of quality assurance. Limitations boilerplate. No sensitivity testing.
- Score 2: No peer review. No sensitivity testing. Limitations absent.
- Score 1: No quality checks or validation.

OUTPUT FORMAT:
Return your assessment as a table followed by a summary:

| Dimension | Score (1-5) | Evidence from Document | Priority Revision |
|-----------|-------------|------------------------|-------------------|
| Analysis Plan-to-Question Mapping | | | |
| Analytical Methods Specification | | | |
| Data Preparation and Integration Plan | | | |
| Triangulation and Synthesis Logic | | | |
| Quality Checks and Validation | | | |

**Total: X/25**
**Band:** Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10)
**Single Most Important Revision:** [One specific sentence]

For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.

DOCUMENT TO SCORE:
[Paste your analysis plan section or full document here]