DQA Scorecard

Score your data quality across five standard dimensions with rubric-based assessment

Validity

Not checked

The extent to which data measures what it is intended to measure.

Indicator definitions are clear and measurable

What the Scores Mean5: All indicators have SMART definitions with specified numerators and denominators where applicable.3: Most indicators have written definitions but some lack specificity in measurement method.1: Indicator definitions are unclear, contradictory, or missing for key indicators.

Data collection tools are appropriate

What the Scores Mean5: Tools have been pilot-tested, validated for the context, and directly aligned to each indicator.3: Tools are designed for the indicators but have not been recently validated or pilot-tested.1: Tools do not clearly measure the intended indicators or have known design issues.

Operational procedures ensure consistent measurement

What the Scores Mean5: Detailed SOPs exist, all staff are trained, and regular calibration exercises are conducted.3: Basic procedures are documented but training is inconsistent or no calibration is done.1: Data collection is ad-hoc with no documented procedures or training.

Reliability

Not checked

The consistency of data collection and the ability to produce the same results under the same conditions.

Results are reproducible across different data collectors

What the Scores Mean5: Inter-rater reliability testing is conducted regularly and agreement rates exceed 90%.3: Spot checks show general consistency but formal reliability testing is not routine.1: Different data collectors produce noticeably different results for the same subjects.

Data entry is consistent and controlled

What the Scores Mean5: Double data entry or automated validation rules catch errors. Error rate is below 1%.3: Single data entry with some validation checks. Occasional errors are found during review.1: No validation rules. Frequent data entry errors and inconsistencies.

Data collection tools are applied consistently

What the Scores Mean5: All sites use the same tools, version control is maintained, and changes are documented.3: Tools are mostly consistent but some sites use older versions or local adaptations.1: Significant variation in tools across sites with no coordination or version control.

Timeliness

Not checked

Whether data are collected, processed, and reported on schedule to support timely decisions.

Data are collected on schedule

What the Scores Mean5: Data collection meets scheduled deadlines consistently (>95% of reporting periods on time).3: Most data collection is on time but delays of 1-2 weeks occur in some reporting periods.1: Data collection is frequently delayed, missing entire reporting periods.

Data are reported promptly after collection

What the Scores Mean5: Data are processed and available for reporting within 1 week of collection.3: Data are available within 2-4 weeks, sometimes causing delayed reports.1: Data processing backlogs mean reports are months behind data collection.

Reporting frequency matches decision-making needs

What the Scores Mean5: Reporting frequency aligns with management review cycles and donor requirements.3: Reporting frequency is adequate but does not always align with when decisions are made.1: Data are reported too infrequently to inform adaptive management decisions.

Precision

Not checked

Whether data have enough detail and accuracy to measure the intended changes.

Data are sufficiently disaggregated

What the Scores Mean5: Data are disaggregated by all relevant categories (sex, age, location, disability) as required.3: Some disaggregation exists but key categories are missing or inconsistently applied.1: Data are collected only as aggregates with no meaningful disaggregation.

Sample sizes are adequate

What the Scores Mean5: Sample sizes are calculated and justified, providing adequate statistical power for analysis.3: Sample sizes are reasonable but not formally calculated or justified.1: Samples are convenience-based or too small to draw meaningful conclusions.

Measurements are precise enough for the intended use

What the Scores Mean5: Measurement precision matches or exceeds what is needed to detect meaningful changes.3: Measurements are generally adequate but may miss small but important changes.1: Measurements are too imprecise to detect expected changes or trends.

Integrity

Not checked

Whether data are protected from unauthorized access, manipulation, or bias.

Access controls protect data from unauthorized changes

What the Scores Mean5: Role-based access controls, password protection, and edit permissions are enforced for all data systems.3: Basic password protection exists but access controls are not consistently applied.1: Data files are openly accessible with no access controls or protection.

Audit trails track data changes

What the Scores Mean5: All data modifications are logged with user, timestamp, and reason. Audit trails are reviewed regularly.3: Some logging exists but audit trails are incomplete or not reviewed.1: No audit trail. Changes to data cannot be traced or verified.

Data are free from systematic bias or manipulation

What the Scores Mean5: Independent verification processes exist. No evidence of systematic bias or data fabrication.3: No known bias issues but no formal verification processes are in place.1: Known concerns about data inflation, fabrication, or systematic bias exist.

Score each criterion 1 to 5

  • ValidityNot checked
  • ReliabilityNot checked
  • TimelinessNot checked
  • PrecisionNot checked
  • IntegrityNot checked
Validity-
Reliability-
Timeliness-
Precision-
Integrity-

Need Help Improving Data Quality? Our advisory services help M&E teams build data quality systems that meet donor requirements and produce trustworthy evidence. See services or talk to us.