Design Evaluator Selection Criteria
Design the evaluator selection criteria and scoring matrix for an evaluation ToR (expertise specificity, experience calibration, scoring transparency, diversity, fairness).
Draft: Design Evaluator Selection Criteria
Fill In Your Details (5 Blanks)
Your Prompt (With Your Details)
This is the complete prompt, ready to copy and paste.
Evidence and applicability: use only supplied or verifiable evidence for historical facts, values, quotations, completed activities and approvals. Preserve conflicting sources with their provenance; do not choose a preferred value without adjudication. Mark missing evidence as unknown or not evidenced, not as proof an activity never occurred. Keep proposed plans and illustrative examples separate from findings. Never invent a baseline, quote, pilot, validation or sign-off to complete a template. Match methods to the purpose and population; a suitable single-method design is valid. Distinguish descriptive change from causal attribution. Check obligations in the applicable funding agreement rather than assuming a universal donor rule.
You are a senior MEAL specialist with procurement experience, designing the evaluator selection criteria and scoring matrix for the Terms of Reference of an evaluation of your program.
The criteria must identify the right candidate on substance, score reproducibly, and withstand bias claims.
Requirements:
- Required expertise specificity. Specify technical skills, sector knowledge (sub-sector, not just "health" or "education"), methods knowledge (qualitative, quantitative, or mixed with named techniques), and tools or software proficiency where relevant.
- Experience breadth and depth. Set minimum years calibrated to scope (e.g., lead vs team), minimum number of similar engagements, geographic experience, and methods experience over a stated recent window.
- Scoring matrix clarity. Build a transparent, weighted matrix that two reviewers applying it would score similarly. Specify dimensions, weights, scoring anchors (1-5 with definitions), and how scores roll up.
- Diversity and GESI considerations. Embed criteria for local expertise, gender balance, language proficiency, and lived experience relevant to the evaluation scope.
- Procurement fairness. Ensure criteria are objective, evidence-based, and consistent with internal procurement policy. Avoid criteria that effectively pre-select a known bidder.
- Disqualifiers and pass/fail elements. Distinguish between scored criteria and mandatory pass/fail requirements (e.g., conflict of interest, registration status).
Output Format:
Produce:
- A 1-paragraph framing of the selection approach.
- A required expertise list (technical, sector, methods, tools) with specificity.
- A scoring matrix table (dimension, weight, scoring anchors 1-5, evidence required).
- A diversity and GESI sub-section.
- A pass/fail requirements list.
- A short note on how the criteria align with the evaluation scope and avoid foreseeable bias claims.
Check: Score It with the Evaluator Selection Criteria Rubric
The Review Prompt
Paste the draft at the bottom of this prompt and run it. It scores each criterion from 1 to 5 and says why.
Use source evidence; retain unknowns/conflicts. N/A or missing evidence means no total. Critical ethics/validity failures block use at any score. Scores never authorize fielding or compliance.
You are an expert M&E procurement specialist. Score the evaluator selection criteria section of the Terms of Reference I will provide using the rubric below.
SCORING RUBRIC - Evaluator Selection Criteria Rate each dimension 1-5:
DIMENSION 1: Required Expertise Specificity
- Score 5: All 4 elements: Required technical skills named specifically (e.g., "quasi-experimental impact evaluation design," "contribution analysis," "qualitative coding with NVivo"), sector knowledge specified (sub-sector, not just "health" or "education"), methods knowledge specified (qualitative, quantitative, or mixed with named techniques), and tools or software proficiency listed where relevant.
- Score 4: At least 3 of 4 elements: Skills and sector named; methods or tools partial.
- Score 3: At least two of four elements present. Generic skills with one specific element. Sector named but broadly.
- Score 2: Skills stated as "M&E experience" or "evaluation expertise" without specificity. No methods or tools named.
- Score 1: No required expertise stated, OR expertise requirements unrelated to the evaluation scope.
DIMENSION 2: Experience Breadth and Depth
- Score 5: All 4 elements: Minimum years of experience stated and calibrated to scope (e.g., 8+ years for lead, 4+ for team members), sector experience specified with minimum number of similar engagements, geographic experience specified (region, country type, or specific countries), and methods experience specified (e.g., "minimum two mixed-methods evaluations in the past five years").
- Score 4: At least 3 of 4 elements: Years and sector calibrated; geographic or methods experience partial.
- Score 3: At least two of four elements present. Years stated but sector or geography generic.
- Score 2: Years stated without scope calibration. No prior-engagement requirements.
- Score 1: No experience requirements stated, OR experience requirements clearly inappropriate to the scope (too low or unrealistically high).
DIMENSION 3: Scoring Matrix Clarity
- Score 5: All 4 elements: Scoring matrix included with named criteria, weights stated per criterion (totaling 100% or a stated total), point scale defined per criterion (e.g., 0-10), and scoring guidance describes what evidence yields what score so two reviewers would converge.
- Score 4: At least 3 of 4 elements: Matrix with weights and scale; reviewer guidance partial.
- Score 3: At least two of four elements present. Matrix included but weights or scale incomplete.
- Score 2: Criteria listed but no matrix, weights, or scoring scale.
- Score 1: No scoring approach described, OR scoring left to undocumented reviewer judgment.
DIMENSION 4: Diversity and GESI Considerations
- Score 5: All 4 elements: Local or in-country expertise requirement specified, gender balance addressed (team composition expectations or explicit consideration), language requirements specified (working languages plus any local languages needed for fieldwork), and lived experience or representation relevant to the program named where appropriate (e.g., disability-led inquiry, youth participation).
- Score 4: At least 3 of 4 elements: Local expertise and language addressed; gender or lived experience partial.
- Score 3: At least two of four elements present. Local or language addressed but gender and lived experience absent.
- Score 2: One element present. Diversity referenced as a checkbox without substance.
- Score 1: No diversity or GESI considerations.
DIMENSION 5: Procurement Fairness
- Score 5: All 4 elements: Criteria are objective (verifiable against CV or proposal evidence), criteria are non-discriminatory (no requirements that exclude qualified candidates without programmatic justification), evaluation process is documented (who reviews, how scores combine, how ties are broken), and feedback to unsuccessful bidders is offered or referenced.
- Score 4: At least 3 of 4 elements: Criteria objective and process documented; non-discrimination or feedback partial.
- Score 3: At least two of four elements present. Criteria objective but process or feedback unstated.
- Score 2: Subjective criteria (e.g., "fits with organization culture") without operational definition. No process.
- Score 1: No process documented, OR criteria contain discriminatory elements with no justification.
OUTPUT FORMAT: Return your assessment as a table followed by a summary:
| Dimension | Score (1-5) | Evidence from Selection Criteria | Priority Revision |
|---|---|---|---|
| Required Expertise Specificity | |||
| Experience Breadth and Depth | |||
| Scoring Matrix Clarity | |||
| Diversity and GESI Considerations | |||
| Procurement Fairness |
Total: X/25 Band: Strong (22-25) / Adequate (17-21) / Needs Revision (11-16) / Substantial Revision (5-10) Single Most Important Revision: [One specific sentence]
For any dimension scored 1 or 2, add a brief explanation and a concrete revision example.
EVALUATOR SELECTION CRITERIA TO SCORE: [Paste your evaluator selection criteria section here]
Fix: Revise What Scored Low
The Revise Prompt
Paste the draft and its scores at the bottom of this prompt. It revises the parts that scored below 4.
Use source evidence; retain unknowns/conflicts. N/A or missing evidence means no total. Critical ethics/validity failures block use at any score. Scores never authorize fielding or compliance.
You are an expert M&E procurement specialist. Review the draft evaluator selection criteria I will provide and fix every problem that falls below the Good (4) threshold on the Evaluator Selection Criteria rubric.
WHAT TO LOOK FOR AND FIX:
Expertise Specificity problems: Generic skills (e.g., "M&E experience," "evaluation expertise") without named methods. Sector named broadly (e.g., "health") without sub-sector. No methods or tools specified.
Experience problems: Years stated without scope calibration. No prior-engagement requirements. Geographic experience absent or generic. Methods experience without recency expectation.
Scoring Matrix problems: No matrix included. Weights absent or do not sum to a stated total. Point scale missing. No reviewer guidance explaining what evidence yields what score.
Diversity and GESI problems: No local expertise requirement. No language requirement beyond working language. No gender balance consideration. Lived experience or relevant representation absent.
Procurement Fairness problems: Subjective criteria ("good fit," "team chemistry") without operational definition. No documented review process. No tie-breaking rules. Discriminatory requirements without programmatic justification.
OUTPUT FORMAT: For each problem found:
- Dimension: [which dimension]
- Location: [which subsection of the selection criteria]
- Problem: [one sentence describing what is wrong]
- Revised version: [the corrected text]
Then provide the complete revised selection criteria incorporating all corrections.
DRAFT EVALUATOR SELECTION CRITERIA TO REVISE: [Paste your draft evaluator selection criteria here]